wPerf: Identifying Critical Waiting in Multi-threaded Applications

In a multi-threaded program, a thread could be executing some tasks or waiting for events. Therefore, when optimizing a multi-threaded program, a developer has two options: one is to identify critical tasks and optimize their code; the other is to identify and reduce critical waiting. Optimizing tasks can indirectly reduce waiting time in many cases, but waiting time may also be directly reduced by techniques like fine-grained locking or non-blocking APIs [1]. To illustrate the problem, we show an example of a badly designed multi-threaded program in Figure 1. Since thread B needs to wait for a response from thread C each time (Figure 1a), B and C actually cannot process requests in parallel at runtime (Figure 1b). As a result, B and C combined take 10ms to process a request, which is longer than the 8ms spent in A. Therefore, in this example, the combination of B and C becomes the bottleneck. Techniques like critical path analysis (CPA) [4, 6–10, 12, 13] and causal profiling [3] can identify correctly that funB and funC are critical tasks, but optimizing funB or funC is not the only option: another option is to make thread B non-blocking. While many existing works have investigated how to find critical pieces of code that are worth optimizing, we find no tools can provide insights about which waiting relationships are important. Currently, identifying such problems relies on developers expertise and will become more challenging when the applications become more complex. The goal of our paper is to develop a systematic method to identify critical waiting relationship whose optimization can lead to significant performance improvement. This paper makes the following contributions: 1. It presents a graph-based model to identify critical waiting in multi-threaded applications. 2. It proposes wPerf, a tool that can build the model for unmodified applications in Linux with 3% overhead on average.